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{SI) Abstract , « ' 

Methods and an apparatus arc described whereby Internet content is coordinated with audio communications, such that two or moi 
parties can view ihe same media content on the Internet while also simultaneously communicating over a traditional telephony network c 
by using voice over network implementations. The visual interaction is achieved by having a user computer display shared content th: 
correspond to a second computer's display, such that both parties will be viewing a copy containing the same content on each of the 
browsers. The visual interaction further provides a method for allowing either of the parties to update the visual content of their browser 
In a further embodiment the resource collection functions of the user computer are enhanced by having the server only download to th 
user computer a user applet containing the information required by the user computer as is necessary. In this way. the user computer dot 
not have to wail as long or dedicate resources to the visual interaction that are not necessary at the time. In a further embodiment Ih 
method works through firewalls on the network. 
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Title: Method and Apparatus for Coordinating 
Internet Multi-Media Content with 
Telephone and Audio Communications 

Background of the Invention 

5 The present inveniion relates generally lo an apparatus and niclhod for 

coordinating Internet communication between two users. More particular]}, 
the invention relates to an apparatus and method for coordinaiinii Internet 
multi-media communication such that the two users can view and modify a 
copy of the same multi-media content, where any modifjcaiions made b\ one 
U» user are subsequently viewed by a second user, as well as enhancmy the 
communication by supplementing any visual communication with audio 
communication. 

Businesses commonly service customers through call centers staffed with 
technical suppon personnel, sales suppon personnel, or service personnel 
15 (hereinafter collectively referred to as sales represfsifative) thai can answer the 
customer's questions. Typically, these centers use an automated call 
distributor ("ACD") system to handle incoming calls from the telephone 
network. Commercially available ACD systems provide call processing- voice 
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processing- data processing and network capabilities for the call centers. The 
ACD system usually routes customer calls to a sales representative. The sales 
representative typically uses a headset and a user device connected to the ACD 
system to respond to the incoming calls. The ACD system can route the calls 
5 to queues, which can be customized to rellect how a business wants to 
prioritize and route incoming calls. 

While the customer is on-hold, the ACD system also pnwides ihe 
customer with information. Typically, the ACD system will notify the 
customer that the customer's call is important, "please hold", suggest that the 
]{) customer call back durmg non-peak hours, or inform the customer the time the 
customer can expect to be on-hold. for example. In such an implementation, 
the customer has to vvait on the telephone while listening to the repeated 
messages of the ACD system. This can frustrate the customer. It also restricts 
the customer from performing many other tasks while the customer is waiting 
15 on the phone for the sales representative or technical specialist. In some 
implementations, the ACD systems pipe advertising to the customer while the 
customer is on-hold. The customer, however, may not be interested in the 
products or services that are being advenised, and may have further frustration 
from being subjected to the advertisements. Alternatively, the ACD system 
20 can provides the customer with melodies during the on-hold period. In this 
implementation, however, the customer is still restricted from doing other tasks 
and the business looses an opportunity to advertise. 

Another way businesses communicate with customers is via hiternet web 
pages. Businesses elicit questions from customers by advertising the 
2S business's products on Internet web pages. Such web pages allow the customer 
requesting that a sales representative of the business contact them, l o request 
service, the customer insens their name and e-mail address or phone number in 
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a space provided on the web page and activates a call-mc button. Thereafter, 
the sales representative is sent an e-mail containing the request for service, lo 
which the sales representative responds. However, when the sales 
representative responds, either by phone or e-maiL the customer is most hkely 
5 no longer viewing the businesses web page, and thus the businesses web page 
cannot contribute to the sales representative's pitch. Moreover, the response is 
not immediate, sometimes witli log times measured in days, and during the 
interim the customer may buy another product. 

An intermediate solution to answer a customer's questions regarding 
10 products is presented by allowing the customer to use or view a demonstration 
of the businesses product while on the Internet. By way of example, if a 
customer wants lo buy software, the customer may go to the business's web 
site and request a software demonstration. The web site downloads to the 
customer the software demonstration. If all the customer's questions are 
15 answered by viewing the demonstration, then the customer can then purchase 
the software. If the customer still has unanswered questions, however, the 
customer will again have to either place a telephone call or wait for a delayed 
e-mail or call response. 

A further disadvantage of providing ihe customer a software 
20 demonstration downloaded from the Internet is the amount of time it takes a 
customer to go to each web site, ascertain the product the customer wants to 
view, and then wait while the appropriate plug-in or code, for example, is 
downloaded to the customer's computer. If the customer only needs lo view a 
part of the demonstration, downloading the entire code lo a customer 
25 unnecessarily uses resources of the customer. In addition, the time the 
customer waits for code to be downloaded is again wasted time, just as was 
waiting on the phone. One solution to this problem is taught in US Patent No. 
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5,572,643 to Judson which claims a method for displaying information on a 
graphic user interface when a customer is waiting for a link to download on the 
Internet. However, this is akin to the making the customer listen to unwanted 
advertisements on the phone and, thus, still has the previously described 
5 problems. 

A iurther disadvantage of downloading a software demonstration is that 
whatever is downloaded is viewed by the customer alone without the assistance 
of the sales representative. The software demonstrations are not usually 
concurrent-use software applications that can be viewed simultaneously at 
!() remote locations. Thus, the software applications cannot be viewed by both the 
customer and the sales representative simultaneously. Further, if they were, 
this would require even longer download periods. In addition, such 
applications would require a large amount of the customer's resources lo run 
and be difficult for the customer to manipulate. 
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Summary 

The aforementioned and other objects are achieved by the invention which 
provides an apparatus for coordinating Inlernel communication between at least 
two users and a method associated therewith. The apparatus provides a visual 
s communication across a network between a user computer and a second 

computer which is in communication with a server disposed between the user 
computer and the second computer. 

The method facilitates visual communication across the network thai is 
platform independent. As such, neither the user computer nor the second 
10 computer is restricted to using any particular operating system or software 
application. The only requirement of the user computer and the second 
computer is that it has a browser-like application for reading files. In the 
preferred embodiment, the browser is a hyper-text markup language ("T ITMI/') 
browser. 

i5 I o provide the visual communication between the user computer and the 

second computer, a link is estabhshed between the user computer and the 
server and a link is established between the server and the second computer. 
T he server, the user computer, and the second computer can be fully distributed 
across the network. 

In addition to being part of the link that is established, the server 
selectively transmits to the user computer a user applet. The user applet 
enables the user computer to communicate with the server. The second 
computer either has a service applet disposed thereon or the service applet is 
provided to the second computer by the server. The service applet and the user 
applet enable the visual communication to occur without requiring either the 
user computer or the second computer to have any executable files pre-insiallcd 



20 



25 
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to enable the user computer or the second computer to have a shared view. 
Further, the user applet and the server applet bi-directionally communicate Wnh 
the server, and thus can bi-direciionaI)y communicate with each other through 
the server. 

5" After the user computer has received the user applet, the user computer 

generates a user view. Either in parallel or thereafter, the server communicates 
that the link is established between the server and the user computer to the 
service applet. Thereafter, a session is selectively established between the user 
computer and the second computer. 

)(i During the session, the second computer and user computer both 

display on the browser of their computer, respectively, a shared view in their 
user interfaces. The shared view allows a user of the user computer to see a 
copy of the same content as is displayed on the second computer, and vise 
versa. Thus, both users of the user computer and second computer will be 

15 looking at the same content displayed on the browser. 

The user interface of the user computer is called a user view, while the 
user interface of the second computer is called a service view\ The service 
view contains a representation of the user view, the representation of the user 
view is the shared view. 

2i) Either the user computer or the second computer can manipulate the 

shared view. For example, the second computer can change the shared view 
from content at one universal resource locator C'URL''), source location to 
content at a second URL source location. If the second computer implements a 
change to the shared view, the change is communicated to the service applet. 

2.^ The service applet transmits the change to the server, and the server 

subsequently transmits the changes to the user applet. The user applet then 
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implements ihe change in the user view, and notifies the server thai the change 
has been implemented. After the sen/ice applet receives notiHcaiion from the 
server that the user view has changed, the service applet changes the 
representation of the user view in the service view, thereby conforming ihc 
> shared view of the second computer to match the shared view of the user 
computer. This process enables the second computer to communicate to the 
user computer. The process can be performed bi-directionaliy, such that the 
user computer can change the user view and cause a conforming change in the 
second computer. 

!0 Various implementations of this method can be enabled by the user 

computer or the second computer. One implementation allows changes to the 
URL source location of the shared view. Another implementation allows 
changes to the images m the shared view. A further implementation of this 
embodiment of the invention allows the shared view to be a software 

15 application. A siil! further embodiment allows the shared view to display 
software demonstration programs. 

An additional implementation of this embodiment is an interactive use of 
forms, where each form has objects. In this implementation, the form is 
represented as an array having arguments where the arguments represent the 
20 objects of the form. As the user computer or the second computer changes 

entries on the form, the user applet or the service applet, respectively, updates 
the objects in the array such that the array now represents any changes in the 
user interface. Then as previously described the changes are implemented in 
the shared view of both the user computer and the second computer. 

25 A further implementation for the interactive use of forms supplements the 

forms to be event monitoring, such that when the objects on the forms change 
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the objects notify the applets of the change. A still further implementation 
rewrites the objects as applets. Again, the changes to the objects are 
implemented in the shared view. 

To enable the visual communication, the server must download the user 
.'^ applet to the user computer. In one embodiment of the present invention, the 
server enhances a resource collection function of the user's computer by only 
downloading to the user computer a user applet representative c^f the 
functionality the user computer needs at that time. As such, the server 
ascertains what the user computer needs and generates an applet representative 
10 of that functionality and downloads that applet to the user computer. If later on 
durmg the visual communication the user computer requires greater 
functionality, the server will then download a second user applet containing the 
necessary functionality. In this way. the server decreases the resource 
collection of the user computer in both time and space while enabling the 
15 functions of the resource that the user wishes to enable. 

The user applet is further enhanced by the use of a script. The script is 
displayed on the user interface of the second computer, preferably adjaceni to 
the representation of the user vjew. The display can be a list of scripts from 
which the second computer can choose or the script that the user of the second 
20 computer has selected. The script is in communication with the service applet. 
The script has simplified commands which it transmits to the service applet to 
enable the service applet to implement any changes in shared view and transmit 
the changes to the user applet as previously described. ,ln the preferred 
embodiment the script is written in hyper-iext markup language ("flTMl/"). 

25 The visual communication can be between more than one user computer 

and more than one second computer. This enables many-to-one, one-to-many. 
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or many-to-many cominunication between at least one user computer and at 
least one second computer. Each of the more than one second computer can be 
identical or each of the more than one second computer can have differing 
service applets which are adapted to perform differing functions, such as 
administrative, sales, or technical support. 

If the embodiment has more than one second computer, the server also has 
more than one queue to which it routes call requests from the user coinputers. 
A queue contains one or more call requests from the user computers. The 
queue is further displayed selectively on one or more second computers. The 

10 server routes call requests using various methods, such as a firsi-come. first- 
serve basis, according to attributes of the user computers, sales specialty of the 
second computers, or logic formulas, which may comprise formulas relate to 
time of call, number of call requests presently in queue, among other items. 
The attributes can be collected by prompting to the user computer, or the user 

15 applet can extract the attributes from the user computer. 

When the call request oi lhe user computer is placed in a queue, the 
server notitles the user applet regarding status of the of the queue user 
computer's call request. The server will display to the user computer in its user 
interface the status of the queue, enabling the user computer to know when his 
20 call request will likely be answered. 

In one embodiment, while an entry of the user computer is in the queue, 
the server through the user applet displays supplemental information which is 
analogous to MUZAK, a trademark of Muzak, Int., on the user computer until 
the user computer's call request is answered. In one embodiment, the 
25 computer supplemental information is selectively transmitted from the server 
based upon the attributes of the user computer. In another embodiment, the 
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server transmits the computer supplemental information based upon scripts. 
While the user computer is being displayed the computer supplemental 
information, the user computer also has a control view generated by the user 
applet that enables the user computer options such as disconnect or call mc 
5 later, among other options. 

In still another embodiment, the visual communication across the 
network is married to audio communication between the user computer and the 
second computer. In one implementation of this embodiment, both the user 
computer and the second computer have a voice-over net capability that is 

10 enabled during the session between user computer and second computer. In 
another embodiment, a traditional telephony system is used to implement the 
audio communication during the session. In this embodiment one audio device 
will be associated with at least one second computer and at least one user 
computer. The audio device can either initiated the communication or be used 

15 alter the mitiation of the visual communication between the user computer and 
the second computer. As such, the users of the user computer and the second 
computer can view shared content over the network as well as audibly 
communicate. 

In the implementation where the audio communication is established first, 
20 the second computer directs the user computer to an appropriate address, or 

URL source location, whereupon the user computer is linked to the server and 
provides a user applet as previously described. In this embodiment, the user 
computer can also selectively be provided a password Vo enter the call session, 
thus enabling the call session to be private. 

2.^^ In still another embodiment, the second computer can be a component ol a 

call center where the call center has an automated call distribution system 
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("ACD"). In this embodiment, the ACD also has a call queue. The call queue 
can be communicaled to the server via the network if the ACD is also 
connected to the network, and as such the call queue of the ACD can work in 
landcm with or complement queue of the server. In this embodiment, the call 
5 center can utilize any additional capabilities of the ACD while supplementing 
the ACD with visual communication enabled by the server, the user applet, 
and the service applet. 

In another embodiment, the communication between the server and the 
user computer can be conducted through a firewall using tunneling. 

to In further aspects, the invention provides methods in accord with the 

apparatus described above. The aforementioned and other aspects of the 
invention are evident in the drawings and in the description that follows. 
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Brief Description of the Drawings 

The foregoing and other objects of this invention, the various features 
thereof, as well as the invention itself, may be more fully understood from the 
following description, when read together with the accompanying drawings in 
3 which: 

Figure 1 shows a system diagram of one embodiment of this invention; 

Figure 2 shows a block diagram of the system of Figure 1 ; 

Figure 3 shows a user interface in accordance with one embodiment lU ihis 
invention; 

10 Figure 4 A shows a second user interface in accordance with one 

embodiment of this invention; 

Figure 4B shows a third user interface oi' another embodiment of this 
invention; 

Figure 5 shows a system diagram having more than one second computer 
15 in accordance with another embodiment oi lhis invention; 

Figure 6 shows a system diagram of yet another embodiment of this 
invention; 

Figure 7 A shows a flow chart of one implementation of h<m' a user 
computer enters a call session in accordance with this invention: and 

20 Figure 7B is a flow chart of a second implementalio'n of how a user 

computer enters a call session in accordance with an embodiment of this 
invention. 
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Detailed Description 

While the present invention retains utility within a variety of networks and 
may be embodied in several different forms, it is advantageously employed in 
connection with the Internet. Although this is the form of the preferred 
5 embodiment and will he described as such, this embodiment should be 
considered illustrated and not restricted. 

Figure 1 shows a system 10 according to one embodiment of the present 
invention. As shown, the system 10 includes a user computer 12 m 
communication with a network 16 and a second computer 24 in 
]() communication with the network 16. The system 1 0 further comprises a server 
20 interposed between the user computer 12 and the second computer 24 also 
in communication with the network 16. 

In the preferred embodiment, network 16 is the Internet. As such, 
communication there over is via Transfer Control Protocol/Internet Protocol 
\> ("TCPMP"). It should be obvious to those skilled in the an. however, that 
network 16 could be an intranet or any network capable of supporting 
communication protocols. 

Similarly, the system 10 can be partially distributed, as opposed to the 
fully distributed depiction of Figure 1 . For instance, the server 20 and the 
20 second computer 24 could be connected to a local area network (''LAN'') or a 
wide area network ("WAN'") which, in turn, is connected to network 16. 

The user computer 24 and the second compiitei- 24 can be a personal 
computer, workstation, terminal, laptop or any device capable of displaying a 
user interface and storing an applet. The user computer 12 and the second 
2.^ computer 24, however, must have a browser 1 8, 28. respectively, where a 
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browser shall be defined herein as any application software or firmware that 
can interpret files to create a view thereon, the files include for example. 
HTML, Javascript. An example of such browser includes, for example, any 
common Internet browser, inter alia, and a Java workstation, inter alia. As 
5 described hereinafter in more detail with reference to Figures 7 A and 7B, the 
user computer 12 and the second computer 24 access the server 20 through the 
browsers 1 8. 28, respectively. The browsers 1 8, 28 are displayed on the user 
computer 12 and the second computer 24 via a display 14, 26, which is a 
cathode ray tube or a liquid crystal display, for example. 

H) Once the user computer 1 2 accesses the server 20 utilizing the browser 1 8. 

the server 20 transmits to the user computer 12 a user applet 22. The user 
applet 22 is adapted to enable the user computer 12 to commvmicatc through 
the network 16 with the server 20. A service applet 30 is disposed on the 
second computer 24 to enable the second computer 24 to communicate with the 

15 server 20. The service applet 30 can either be transmitted to the second 

computer 24 by the server 20 or be disposed thereon. After the user computer 
12 receives the user applet 22, the second computer 24 is notified that a link is 
established, and that a session can be established between the user computer 12 
and the second computer 24. When the session is established, users of the user 

20 computer 12 and the second computer 24 can visually communicate. 

The users of the user computer 12 and the second computer 24 visually 
communicate by sharing content on their respective displays 14. 26. More 
particularly, the browser 1 8 displays a user interface containing a user view on 
the user computer 12, while the browser 28 displays a user interface on the 
25 second computer 24. When a session is enabled, the user interface of the 

second computer 24 contains a representation of the user view that is displayed 
on the user computer 12. Thus during a session, the user computer 12 and the 
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second computer 24 arc viewing a copy of the same user vicvv on their 
respective browsers 1 8, 28. The user computer 1 2 and the second computer 24 
are then viewing shared content. 

During the session, either party can manipulate the shared content 
5 displayed on the party's computer. Wlien the shared content is manipulated, 
the applet disposed on that party's computer will communicate to the server 20 
through the network 16 that there has been a change to the shared conicni. In 
the preferred embodiment, the applets 22, 30 selectively poll the server 20 
during a session to ascertain whether an event has occurred, such as a change in 

!0 the shared content. This is facilitated by designating each session and eacli 
applet in the session with a unique identifier, as further described with 
reference lo Figures 7A and 7B- When an applet polls the server 20 having the 
proper session idenlifjer, the server 20 communicates lo the applet thai an event 
has occurred- That applet then communicates with the browser of its respective 

15 computer and causes the browser to display the any changes in the shared 
content. Once the browser of the unchanged computer displays what was 
communicated to it by ihe server 20. both parties will again be looking at ihe 
same shared content on their respective browsers. This process continues back 
and forth such that either party may lead the other to view, for instance, another 

20 web page or a software demonstration, among other items hereinafter 
discussed. 

In another implementation, the server 20, instead of the applet, can retrieve 
the changes to the shared content and then transmit the changes to the applets 
in the session. In this implementation, the server 20 acts as a proxy server, the 
25 function of a proxy server is well known in the art. 
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In the preferred embodimenl, after a user of the second computer 24 elects 
to alter the shared content, the service applet 30 waits for a response from the 
server 20 before updating the shared content on the second computer. It wails 
for a response from the server 20 that it has communicated the user applet 22 
s and has changed the shared content on the user computer 12. Once the server 
20 receives communication back from the user applet 22 that it has chan[^ed the 
shared content on the browser of the user computer 1 2, the server notifies the 
service applet 30 of the second computer 24. At this time, service applet 22 of 
the second computer 24 updates the representation of the shared content 52 of 
10 the second computer\s interface. In this manner, the user interface of the 

second computer 24 will not get ahead of the user computer 12. It should also 
be noted that each computer 12. 24 are retrieving the shared eoniem 52 
independently and, in this instance, not sending the shared content 52 trom one 
computer to another. 

15 h should be obvious lo those skilled in the art that the server 20 can 

inform the applets 22, 30 in each session of any changes without requiring the 
applets 22. 30 to poll the server 20 without departing from the scope of this 
invention. Further, the applets 22. 30 can work in conjunction with the server 
30 such that any combination of polls or pushes results in the applets 22. 30 

20 being notified that an event has occurred. Moreover, polling can be replaced 
with event driven technology, or similar notification technologies, as the event 
driven technology becomes more acceptable on the Internet. 

In the preferred embodiment, the server 20 is a JAVA server utilizing 
,)AVA objects. JAVA is a registered trademark of Sun Microsystems. Inc. The 
25 user applet 22 and the service applet 30 communicate with the objects disposed 
upon the server 20. In the preferred embodiment, the applets 34 are also JAVA 
applets. This enables the system 10 to be a cross-platform system and as such 
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does noi need any executable files previously disposed on the user computer ]2 
or any of the computers involved in the session. 1'he JAVA server can run on 
a UNIX computer, a MACIN TOSH computer, MS Windows computer, a 
Silicon Graphics computer, or any other well-known computers in the an. 
s UNIX is a registered trademark of AT&T Corp. MACINTOSH is a registered 
trademark of Apple, Inc. 

The user applet 22 performs event monitoring and communication as was 
described in part above. The user applet 22 also has a model controller view. 
1 he model controller view of the user applet 22 is the user view which is 

H) further described w^th reference to Figures 4 A and 4B. The user applet 22 also 
shares much of the object model of the system 10. unlike a typical client. The 
user applet 22 having a portion of the object model allows the system 10 to 
function when ii is fully distributed. This implementation makes it easy to 
replace the polling mechanism with distributed object communication 

15 technologies including, for example. Remote Method Invocation ("RMP') © 
Javascript. Sun. Common Object Request Broker Architecture ("COBRA"). 
Internet Inter-Orb Protocol f 'HOP") and Distributed Component Object Model 
("'DCOM''), ® Microsoft. For example, the system 1 0 can manipulate an 
object on the user computer 12 and have the user computer 12 transmit through 

20 the user applet 22 to the server 20 a response, both while the user computer 1 2 
is unaware that the communication or manipulation of the object has occurred. 
This implementation, among other things, enables the server 20 to extract from 
the user computer 12 attributes of the user computer 12 where the attributes can 
include items such as the user computcr\s e-mail «a'ddress. the user computer's 

25 history of links on the network 16, and the user computer's specifications, 
among other items. 
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In the preferred cmbodimcnU ihe user applet 22 is a persistent applet. The 
user applet 22 is loaded into a cache (not shown) of the user computer 12, 
preferably a local cache. It is persistent because once the user applet 22 is 
downloaded from the server 20, the user applet 22 remains on the user 

s computer 12 as long as the user computer 12 remains in the session with the 
server 20. unless it gets removed from the local cache by the user computer 12 
as is well known in the art. However, if it is not removed, the persistent applet 
will remain disposed on the user computer 12 such that it will not have to be 
downloaded again from the server 20. The user applet 22 will remain in the 

H) cache regardless of any changes to the user interface of the user computer 1 2. 

The user applet 22 is selectively generated by the server 20 to enhance a 
resource collection function of the user computer 12 while enabling the user 
computer 12 lo participate in the visual communication, 'fo enhance the 
resource collection function of the user computer 12, the server 20 evaluates 

!5 the functions of a resource, such as a spreadsheet application for example, that 
the user computer 12 requires at any given point in time. After the server 20 
evaluates these functions, it then generates the user applet 22 representative n\' 
the functions. Once the server 20 generates the user applet 22 representative of 
these functions, it transmits the user applet 22 to the tiser computer 1 2. as 

20 previously described. 

Typically, the functions represented in the user applet 22 are only a subset 
of the functionality of the resource. This is analogous to using EXCEL, a 
trademark of Microsoft Corp., spreadsheet and only getting the graphing 
feature when one tries to use it, for example. By only including a subset of the 
25 functionality as required by the user computer 12, the time for downloading 
between the server 20 and the user computer 12 is decreased. Further, the 
amount of storage space required in the user computer 12 is also decreased. 
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1 hus, the server 20 by selectively generating the user applet 22 enhances the 
resource collection function of the user computer 12. It should he obvious to 
those skilled in the art that the server 20 can, in real-time or pursuant to a timer, 
continually download to the user computer 12 any necessary functionality in an 
5 applet 22- The user computer 12 can also trigger the server 20 to inform it that 
it needs to be given an additional applet. In the system 1 0, the user computer 
12 receives what it needs as it needs it. Thus, the user computer 12 is 
optimized as a very thin client in the thin client fat server model. 

For example, if a user computer 12 enters a session to share a web page as 
10 further described hereinafter, the server 20 only downloads functionality lo the 
user computer 12 necessary for the browser I 8 to be notified about, and to 
display the web page. If. for example, the user computer 12 is sharing a 
software application, then the server 20 will download to the user computer 12 
another applet in real time of the next piece of functionality the user computer 
15 12 needs lo implement the software application. 

It should be obvious to those skilled in the art that the selective generation 
of an applet and subsequent downloading of that applet can be used for 
applications outside of the system 10 withoul departing from the scope of this 
invention. 

20 Turning now to the service applet 30, the service applet 30 can comprise 

one or more applets each adapted to provide the second computer 24 with 
different functionality. The service applet 30 at minimum must provide the 
second computer 24 with the ability to communidaie with the server 20. 
Again, the server 20 can download the service applet 30 in real time to the 

25 second computer 24 during the session. The service applet 30 shares much of 
the object model with the user applet 22, as previously described, to enable 
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jmplemcntation ot the system 10 in a distributed environment. One 
implementation uses a proprietary interface definition language to 
communicaie session inlbrmation between applets 22. 30 and the server 20 
based on a polling model, while another is based on event notification. Thus, 
5 the service applet 30 and the user applet 22 enable the user computer 12 and 
the second computer 24 to bi-directionally communicate over the network 16. 

The visual communication of system 10 is further enhanced by audio 
communication between the users of the user computer 1 2 and the second 
computer 24. In one implementation, the user computer 12 connects to the 

10 server 20, receives the user applet 22. and the service applet 30 is notified of a 
call request from the user computer 12. Upon notification of the request, the 
second computer 24 implements audio communication either through voice 
over network ("VON") applications or a traditional telephony system tnot 
shown). At this point, the user computer 12 and the second computer 24 are m 

15 communication both audibly and visually. 

The visual communication can be turlher described with reference to 
Figure 2, which shows a block diagram of the system of Figure 1 . As was 
previously described, the server 20 transmits through the network 16 applets 
34. The applets include the user applet 22, the service applet 30, any other 
20 applet necessary to give the user computer 1 2 or the second computer 24 
functionality. 

f he server 20 is capable of ascertaining, as hereinafter described, what 
type of computer is requesting a download of one of the' applets 34. This is 
achieved by having participants, such as the user computer and the second 
25 computer, in a session use different sign-on pages and/or by analyzing an 
information payload sent by the browsers in an H^fTP sign-on request. As 



AVO 97/42728 



- 21 - 



PCT/US97/07740 



such, if the server 20 identifies that the requester is the user computer 12 then it 
downloads the user applet 22 capable of generating a user view 90 and the 
functionality the user computer 12 requires as previously described. If, on the 
other hand, the server 20 ascertains that the requester is a sales representative, 
s then it downloads a service applet 30 capable of generating a sales view 50. 
Similarly, if the server 20 ascertains that the requester is a system 
administrator, the server 20 downloads an applet capable of generating an 
administrator view 36. 

The server 20 ascertains and validates which type of user is requesting one 
of the applets 34 by various methods. One such method allows the server 20 to 
validate each of the computers using a password system. In this 
implementation, any requester can get any view 36, 50. 90 as long as it has the 
appropriate validation codes such as a name and a password. Additionally, 
with the appropriate validation codes a requester can log-c^n from anywhere in 
the world that is connected to the network 1 6. Thus, others can join in a 
current session simply by knowing the validation codes. 

In the preferred embodiment, the sales representative and the administrator 
have predetermined passwords stored in a database (not shown) 
20 communicating with the server 20. The database can be disposed on the server 
20 or remotely therefrom. The server 20 communicates with the database 
through a database interface that uses Java Database Comiectivity {''JDBC'l or 
other Object Database Connectivity (''ODBC') equivalent database languages, 
for example. In the preferred embodiment the danabase is a relational database. 
25 although it can be an object database or other forms of data storage well known 
in the art. 
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As such, when a requester log-on to the server 20, the server 20 checks the 
database to see whether the requester has a valid log-on and the associated 
applets 34 required by the requester. Once the server 20 validates the log-on, 
then as described, the server 20 will download one of the applets 34. In this 
5 way, the administrator can function as a sales representative having at one 
time a service applet 30 downloaded to it via the server 20 and at another time 
have an administrator applet downloaded to it via the server 20. 

The user computer 12 communicates with the server 20 through a web 
page. The user computer 12 either requests a call through the web page or 
!{) enters a password on the web page that was provided lo the user computer by 
the sales representative, as described further with reference to Figures 7A and 
7B. Thus, the server 20 can uniquely identify that a user computer 12 is 
requesting a user applet to generate the user view 90 by the method of 
communication with the server 20. 

1 5 Each of the applets 34 allows the browser of the respective computer to 

generate the associated views 36, 50. 90. In a session, the shared content is 
displavcd in the user view 90 of the user computer 12 by the browser IK. while 
the shared content is displayed on the second computer 24 within the sales 
view 50. fhe user interface of the second computer 24 or the sales view 50 

20 contains a reprcsenialion of the user view 90. 

When the shared content in the user view 90 or the representation of the 
user view is manipulated by one party, through communication among the 
applets 34 and the server 20 the shared content of the otRer parly's view will be 
updated. For example, suppose the sales view 50 contains an H I MF File 
25 located at the URF source location uww.webline.com. The user view 90 will 
also contain a copy of the HTML File with the URL source location 
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wavw.weblinc.com. If ihe second computer 24 wishes to direct the user 
computer 12 to the service's page of the Webline Corporation located at 
www. wehline_services.com, then the second computer 24 can change its sales 
view 90 such that the URL source location of the shared view is 
s \vww.webline_ser\'iccs-Com. Once the second computer 24 changes the sales 
view 50 by methods such as the script as described hereinafter, or just by 
typing in the URL source location, the URL source locati on of that page is then 
transmitted by the service applet 30 to the server 20. The server 20 then tells 
every other applet that is part of the session that there is a new URL source 

If) location. In this example, the server 20 will tell the user applet 22 that there is 
a new URL source location. After the user applet 22 receives information, it 
then causes the browser 1 8 of the user computer 1 2 to pull down a new page 
from the network 16 and display on the user view 90 a copy of ihe HI ML file 
at www. webline services.com. The shared content can include web pages. 

15 flies, application images, advertisements, interactive forms data, or application 
data among any other form of data that can be captured and displayed on the 
browser. 

In addition to the sales view^ 50 and user view 90. other views can also be 
implemented by the applet such as an administrative view 36. for example. 

20 The administrative view 36 displays information such as the structure of the 
sales force, reports, real-time monitoring of the communications, queue 
structure and status, which will be further described hereinafter, for example. 
It should obvious to one skilled in the art that the administration view 36. the 
sales view 50, and the customer view 90 can be disposed on one computer or 

25 more than one computer. 

Further any computer displaying the views 50, 90, 36 require a particular 
operating system 38 to accept the applet and share content in the preferred 
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embodiment. As such, they can be Windows-based. Macinlosh-based, or 
IjNIX-based, ainong others. Further, the browsers 40 can be any number of 
types, such as Netscape Navigator, Microsoft IE. Sun 1 lot Java, for example. 
Further, the browsers 1 8, 28 do not have to be the same on the user computer 
5 1 2 and the second computer 24. 

Turning now to Figure 3, which shows at least one user interface that in 
the preferred embodiment is the sales view 50. hi addition to a representation 
of the user view 52 thai contains the shared content, (he user interface also 
contains a script screen 54. 

0 The script screen 54 can display a plurality of scripts 58 or just one of the 

plurality scripts thai is activated in the user interface. The second computer 24 
can have predetermined scripts 58 illustrated on script screen 54 from which to 
choose, where each of the scripts 58 may be tailored to assist a particular type 
of user computer 12 or second computer 24. For example the scripts 58 could 

5 include: information regarding a company, a stale of the market script, features 
of products, benefits, architecture or any set of information limited only by the 
imagination of the user. In the preferred embodiment, the scripts 58 are written 
in HTML so that the scripts 58 arc displayed in the browser. 

The scripts 58 dictate the shared content contained in the representation of 
0 the user view 52. The scripts 58 perform this function by having links that 
push the shared content into the representation of the user view 52. which, as 
previously described, is also pushed into the user view of the user computer 12. 
The scripts 58 perform this function by communicating'with the service applet 
30. As the service applet in the preferred embodiment is a Java applet, the 
5 script communicates with the service applet 30 using Javascripi-io-Java 

communication in which a Javascript command can invoke a public method in 
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a Java applet. It should be obvious to one skilled in the art that other 
embodiments are possible. Similarly, in another implementation the scripts 38 
can communicate directly with the server 20. instead of the service applet 30. 

The scripts 58 control what is displayed in a section of the browser 
s adjacent to the scripts. As such, the scripts 58 update the browsers in a manner 
different than that is currently done for HTML files. 

An example of a scripl in the preferred embodiment is as follows: 

<htm]> 
10 <head> 

<titie> Web. Dialog Sales Screen</title> 

</head> 

<body> 

<font size=5 color=brown><b> Web. Dialog Sales Screen </b> </font> 
15 <br> 

<fonl size-=2> <b> Web. Dialog version 1 .0 Beta 1 </b> </font> 
<hr> 

<font sizc~4> 
<ul> 
20 <li><a 

href= "javascript :top.saIesrepFramc. document. SalesRep Applet, set Address 
( M"ittp://w\\^\web!inexom') ">Share vavu'. wcbline.com </a></Ii> 
<br><br> 
<li><a 

25 href= * javascript :top.salesrepFrame.document-SalesRepAppiet. set Address 

rhttp://www. microsoft. com') ''>Share www.microsoft.com </a></ii. 
<br><br> 
<li><a 

href= ''javascript: top. salesrepFrame. document. SalesRep Applet, set Address 
30 ('http://w\\av. netscape. com') ">Share ww^. netscape. com </a></li> 
</ul> 
</font> 
</body> 
</html> 

35 In this example, the script is an HTML -lava script, which contains 

Javascript to communicate with the Java applet. The script is displayed in the 
lop sales representative frame of the user interface, where the top sales 
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representative frame is the script frame 54, In the code 
hrcf=^ Javascript: top. salesrepFrame. document. SalesRepApplet. set Address. 
setAddrcss is a public method in the Java applet, which takes as an argument a 
URL source iocaiion, that instructs the applet to share the URL source location 
5 between all the participants in the session, being the user computer 12 and the 
second computer 24 in one embodiment. This particular script, if enabled by 
the second computer 24, will allow the second computer 24 push into the 
shared content the HTML file having a URL source location 
htlp://ww\v.web!ine.c()m. http://wwv.'. microsoft. com or 
10 http://wavw.nelscape.coni. 

For example, if the user of the second computer 24 wants to push 
http://wwv.'.webIinexom into the shared content. In this example, the service 
applet 30 will receive from the script that the second computer 24 wants the 
shared content to display the file having the URL source location 

13 http://wwu .webline.com. The service applet 30 will then waits for a response 
from the server 20 that the server has verified that the user applet 22 has 
instructed the browser of the user computer 12 lo download from the network 
16 the flic having the URL source location http://www.weblinc.com. Once the 
service applet 30 is notified that the server 20 has received the address of the 

20 shared content, the service applet 30 of the second computer 24 updates the 

representation of the shared content 52 of the second computer's interface, and 
the user applet 22 would receive the address of the shared content and update 
the shared content. The process would be repeated if the second computer 24 
then selected to push the www.microsoft.com file into tfie shared content. 

23 It should be obvious to those skilled in the art that scripts 58 can be used to 

push all the forms of shared content herein discussed to the user computer. 
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including, scripts 58 thai initiate a file transfer to download informalion to the 
user view from a remote location, for example. 

As the scripts 58 are in HTML, users can easily write them. vScripts 58 can 
even be written interactively by allowing the user of the second computer 24 to 
s insert the arguments from the script by simply going from one page to the next 
while having the script translate URL of that web page and insert it as an 
argument in its script. The scripts 58 can also be dynamically defined to push 
shared content according to attributes of the user computer 12 and/or attributes 
of the second computer 24. As such, the scripts 58 can be dynamically defu'jed 
!0 by the server 20 that access the attributes. 

In addition, the second computer 24 can dynamically generate scripts 
during the session to meet any requirements of the session. For example, the 
second computer 24 can create a script to push the product comparison into the 
shared content in response to a request of the user computer 1 2 to see a 

15 product comparison. For instance, the second computer 24 can search for 
mformaiion regarding products, by keywords, URL source locations, for 
example. After selecting the informalion the second computer 24 wants to 
push into the shared content and thus the content of the script, the second 
computer 24 can activate the script so generated. This selection can be 

20 implemented by default or by allowing the second computer to chck a generate 
script button or an accept button displayed in a session box 62. for example. 
Again, this would allow the user computer 1 2 to view the product comparison. 
Any comparison information can be displayed to ih^. user computer 12 in this 
manner. Similarly, presentations regarding other information can be conducted 

25 by dynamically generating scripts and then pushing the shared content to the 
user computer 1 2. 
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Adjacent to the script screen 54 and disposed below it is a user information 
screen 56. The user inrormation screen 56 contains queue information 60 and a 
session box 62. 1 he session box 62 contains information about the session in 
which the second computer 24 is engaged in currently. The session inlormation 
5 can include items, for example, duration of the session, address currently being 
displayed in the representation of the shared content 52, any password 
associated with the session, status of the session, ACD interaction information, 
customer information and control functions such as connect 64, hang-up 66, 
pause, new view, log-off, help, among others. 

10 The session box 62 augments the scripts 58 by also providing a URL box 

67. Instead of using a script link, the second computer 24 can insert any URL 
into the L^RL box 67. enable the connect 66 button, and push the shared content 
to the user computer as previously described. 

The second computer 24 is also part of a queue of the server 20. and as 
I S such status of the queue information 60 is displayed on the user mterface 50. 
The system 10 has at least one queue, where each queue contains one or more 
call requests from user computers. In the simplest implementation, when a 
user computer 12 liiiks to the server 20. the server 20 places a call request from 
the user computer 12 in the queue in the order the server 20 received the 
20 request. 

'fhe call request can also be routed by the server 20 into queues based 
upon attributes of the user computer 12, where attributes of the user computer 
12 can be an e-mail address. HTML pages that the user Computer 12 was 
viewing when it made the call request or interests, or database entries for a 
25 user, for example. The server 20 selectively places a call request of the user 
computer 1 2 by parsing the attributes into one or more of the queues. For 
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example, if the user attribute is thai the last page the user computer 12 viewed 
advertised laptops, the server could route the call request to a queue for laptop 
computers. The service applet 30 would then poll the server 20. collect the 
queue information and thereafter display it on the user interface 50 if the 
5 second computer 24 was assigned to the laptop queue. 

The assignments of the second computers are stored on the database 
disposed on the server 20, as well as information regarding the system, such as 
how long does it usually take for each call to be answered, for example. 1 he 
database also contains information collected by the server 20 such as how 
fO many second computers are currently logged-on. how many second compuiers 
are in each queue, how many call requests are already in each queue, whether 
any second computers 24 are on breaks and how long or short breaks, and 
whether more than one second computer 24 should be signed off. 

In one embodiment, the server 20 utihzes the database information and 
15 logic rules in association with the attributes of the user computer to route the 

calls to the queues. For instance, the server 20 could route a user computer to a 
secondary laptop queue, perhaps one with Jess experienced sales 
representatives, when the primary laptop queue had a predetermined number oi* 
call requests. In this example the server 20 could also download an 
20 administrative applet to the administrative view to update the queues 

information 60 displayed on that view, or the server could cause a beeper to 
ring, for example. The server 20 can also compare the attributes, such as name, 
against a database of customers, to enable priority ^ojstomers to be routed 
accordingly. 

25 The attributes could be collected from the user computer 12 using various 

methods. First, the user computer 12 could be prompted to enter the 
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information directly, or by choosing from a drop down list of predefined 
attributes, such as novice user, experienced user, for example. In still another 
embodiment, the attributes can be extracted from the user computer 12 

The service applet 30 retrieves the queue information to display using an 
5 interface language that includes commands, for example, such as, for example, 
tell me how many other are call requests are in the queue, it should be obvious 
to those skilled in the service applet 30 can implement polling or distributed 
object communication technologies without departing from the scope of this 
invention, fhe service applet 30 generates the queue information box 60, 
10 unlike the user applet 22. 

A user computer 12 receives status regarding its position in the queue 
through the server 20, as well as receiving supplemental information, where the 
supplemental information can be analogized to MUSAK, a trademark of 
Musak, inc. The supplemental information can be general information or 

)5 specialized information displayed for the user computer 12. 1 he server 20 
ascertains which specialized information to display to the user computer 12 
using the attribute information of the user computer 12. For example, if the 
attribute of the user computer 12 is laptops, the supplemental information 
displayed to the user computer 12 while in the queue can be advertisements. 

20 accessories or specifications laptops, for example, among other information 

that may be of interest to a user who wants to purchase a laptop. The server 20 
implements displaying the supplemental information by communicating with 
the user applet 22. » 

After the session is established and the user is out of the queue, the user 
>5 view 90 will display the shared content 52. In addition to the sharing a file as 
previously described, the user computer 12 and the second computer 24 can 
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share a demonstration. To share a demonstration, either the user computer 12 
or the second computer 24 must have the appHcation capable of running tlie 
demonstration on the computer, in the preferred embodiment the second 
computer 24 has the application while the user computer 12 only has a 
3 browser. As the software demonstration runs, the second computer 24 captures 
an image of the demonstration, Hkc a print screen, and transmits the captured 
imayc to the user applet 22. Vhc captured image is saved as a jpeg, gif, bmp 
file or other graphical utility, transmitted to the server 20 or the user computer 
12 and displayed on the user computer 12. 

H) The user computer 12 and the second computer 24 can also share a form 

The form has a URL source location 70 and a place to enter user information 
74 and some text 78. In ihe preferred embodiment, the form can be any H TML 
page. The service applet 30 parses the form and stores each of the entries in 
the tbrm as objects, such that the URL source location 70 will be an object, the 

15 text 78 would be an object and user information 74 would be an object, for 

example. In one implementation, the service applet 30 stores the objects as an 
array. In this example, it would be a tw^o dimensional arra\ of three objects. 
Either the objects that have been changed can be transmitted by service applet 
30 to the server 20. or the array containmg all the objects can be transmitted by 

20 the service applet 30 to the server 20, whereupon the user applet 22 would 
parse the array to ascertain which objects had changed. 

If the sales representative had the user enter the text siring 586SX into the 
user information 74 when the user was purchasing. ;>oftware for example, w hen 
the user hit enter, the user applet 30 would be informed that one oi^the objects 
25 of the form in its array had been changed, particularly that the array displa\ ing 
the user inibrmation would now contain a text strjng. The user applet 30 
would communicate this to the server 20 which would then let all the applets in 
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the session, in this case the service applet 22. 'J he service applet 22 would then 
evaluate each object in the array and ascertain which object had changed- After 
identifying thai the user input 74 had changed to now include a text string 
386SX it would update the shared content. 

Figure 4A shows the user view used in the previous example, where the 
user view 00 having the shared view 92. The shared view 92 is esseniial]\- n 
child browser spawned by the user applet 22 to display HTML files. The 
shared view 92 in this example has the URL 96, the text 98, the user 
information 1 00 and the text st rmg 586SX 102. In tins manner, other llie user 
computer 1 2 or the second computer 24 can interactively enter information 
the form that will be displayed to the other party in a real time as prcviouslv 
described. Thus, instead of having the second computer 24 merely push the 
appropriate form to the user computer 12, the user could complete the form 
while in the session and the sales representative could audibly communicate to 
the user whether the user was properly or improperly completing the form. 

In the previous example, the applet was informed that there was an update 
ol'the object by either the user computer 12 or the second computer 24 hitung 
enter. This is not necessary for other entries on the form such as a check the 
box entry. For such entries, the user applet 22 could simply have two values, 
true and false represented by zeros and ones. Each time a check w^as formed in 
the box. the user applet 22 could inlbrm the server 20 of the change in the 
object. The server 20 would then inform all of the applets in the session that 
there has been a change and appropriately update the representation of the 
shared content to include the check. As such, any of the forms on the Internet 
are capable ol' being completed in this manner. 



wo 97/42728 



PCT/US97/07740 



In another embodiment, the form could also be a two dimensional array 
where in each frame of a form is the first argument, and the second argument is 
the objects on each form. In this embodiment, the first frame would be 
designated frame one, the second would be designated as frame two and the 
3 third designated as frame three. 

In still another embodiment, the form could contain minor !nodifications 
that allow it to notify the service applet 30 or the user applet 22 when a change 
occurs in a field, similar to event monitoring, in still a furtlier embodiment, 
applets can be generated to represent the objects of the ibrm. In this 
10 embodiment the applets representative of the objects would notify the user 
applet 22 or the service applet 30. as the case may be of changes to the form. 
In the preferred implementation, the applets would be Java applets. 

If the user computer 12 is attached to a server that disallows an outside 
server 20 to read information from a form, the user computer 12 can enable 
15 services such as data tainting in Netscape Navigator or other well knowMi 

methods to enable the server to see the changes in the form. ]n data tainting, 
the user computer 12 explicitly gives permission to read the information from 
the document that the computer is viewing. 

hi addinon to the shared view 92. the user interface 90 also has a control 
20 view 94. which is framed adjacent to the shared view 92. The control view 94 
allows the user computer 12 to manipulate the session and can include features 
such as, for example, send 104, a box to type an URL location, as well as 
disconnect. The send 104 on the control view 94 'can be selectively disabled by 
the second computer 24. to allow the second computer 24 to control the 
25 interactive visual session when a user computer 12 is manipulating the shared 
conieni improperly. 
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Turning now to Figure 4B, where like numerals denote previouslv 
described elements, there is also shown a second embodiment of the user 
inlcrlace 90". The user interface 90" that contains a control view 94' and a 
shared view 92\ however, in this embodiment, the shared view overlies the 
5 control view and is not framed within the browser 1 8 of the user computer 1 2. 
This is the preferred embodiment as web pages are more commonly displayed 
as one page in a browser and not as frames so that each page can be 
mdividually book marked and print as users are accustomed to. Further, as the 
user computer 12 docs not have to see in the control view it is not necessary to 
n» frame the web pages shown in Figure 4A. In particular, as there is no c|ueue 

information m u.ser view regarding its queue once a session has established, the 
u.ser computer does not need to have a framed page as shown in Figure 3. 

The .shared view 92" of Figure 4B, displays shared content of an 
application wherein the application displayed is a spread-sheet 120 having 

15 menu controls 122. I o share an application, one of the computers in the 

session must have the application running on its computer. In the preferred 
embodiment the application is running on the second computer 24. If the user 
computer 1 2 is the one who warned to share the page, they could select sharing 
the excel page by activating a button or press a hard key. for example, that 

0 stales -Send to Webline". When either party hits the send button, an 

implemeniaiion grabs the screen as an image and then does an File I ransfer 
Protocol (■•FTP")or Hyper-text Transfer Protocol ("HTTP") to put it on a drive 
of the server 20. It should be obvious that there are other ways to implement 
application sharing that may be incomorated into the invention without 
deviating from the scope of the invention. In another embodiment, the shared 
application could be running on the server 20 or any server connected to the 
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network 16. with the second computer 24 and the user computer 12 both 
sharing the same view of the application. 

In ihc preferred embodiment a plug in for Netscape Navigator or an active 
X-control adapted for a Microsoft IE brovv^scr is used. In this cmhodin-teni. 

5 when the user hits the send button the Active X control grabs the screen image 
using Windows API functions or similar functions well known in the art. 
However, once the Java server receives the image, it notifies the applets in the 
session that it has received the image. The current image to he shared is given 
a random and unique URL location which is then sent to the other computer's 

M) applet across the network 16 just as if they were sharing a URL of a web page. 
The process is repeated for each image of the application that the panics want 
to share. 

Image sharing can also be implemented by just transmitted the changes 
between one image and the next, and thus reserve resources on the server. 
1:^ Lither way. the user computer 12 and the second computer 24 can share an 
application without requiring the user computer to be actively running the 
application. 

In a further embodiment, any activity (e.g. key clicks, mouse clicks 
etc.) on the shared view of the appHcation can be transmitted to the computer 
20 running the application and any resulting new changes can be shared. 

Turning now to Figure 5. where like numerals denote previously 
described elements, which shows one embodiment ^of the present invention 
wherein more than one second computer 24. 1 50 is connected 10 a LAN 1 52 
which is then connected to the server 20. Computer 1 50 can be identical to the 
25 second computer 24 having an identical service applet 30 or it can be different. 
It can contain an administrative applet, for example. 
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This system allows ihc second computer 24 to have computer I 50 join a 
session upon request by placing in a queue display 158 of computer 150 a 
request to join the session. When computer 150 activates the connect 160 
feature in the queue display 158, the server 20 through the LAN 152 would 
5 download to the computer 150 the relevant session information as well as an 
applet, if necessary. Ii should be obvious to those skilled in the an that the 
second computer 24 can also transfer the session \n computer I 50 and not jusi 
add computer 150 to the session. 

The server 20 is again connected to the network 16 ior communication 
I (J thereon, and the user computer 12 is connected to the network 16. 

A phone 164 of the user is explicitly show c(^nnected to a public switched 
telephone network CT>STN^^) 166. As such, the user 162 con.stiiuies both the 
phone 64 and user computer 12, The second computer 24 also has a direct 
connection to the PSTN 166 either through a local exchange or a long distance 
i ^ service. As such, when the second computer 24 is informed by the server 20 
thai a call request from user 162 is in its queue, the second computer can 
connect through the public telephone network to the local PSTN 166 ot the 
user 162 and to user's phone 164. 

In this embodiment, the server has a database ] 54 disposed thereon. As 
:o previously described, the database stores information regardmg the queue, 
assignments, attributes, and password validations among other items. 

Turning now to Figure 6, where like numerals denpte previouslv described 
elements, which shows another embodiment of the present invention wherein 
more than one user 162. 164 is shown. User 162 has the user computer 12 and 
2.^ the phone 1 64 as was previously described. User 1 64 can also have a user 

computer 166 and a phone 168. Whcrem both user computers 12. \h6 are in 
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communicaiion vviih network 16 and both user phones 164. 168 are in 
communicaiion with a PSTN 166. The second computer 24 can enter a call 
session with user computer 166 as well as enable the user computer 12 to enter 
the same call session. 

In this embodiment, a ijales center 188 is shown comprising the 
salesperson 190 having the second computer 24 and a phone 180. 'I'he phone 
180, however, is connected to an automated call distribution svslcm ("ACD"') 
1 82. ACDs are well known in the art, and Ihey have, among other features, 
more than one queue 188 disposed thereon within which to route calls Irom the 
publicly switched network or data networks, in this embodiment, when user 
162 enters a call request through the phone 164. the call js transmitted to the 
PSTN 1 66 and then to the ACD 1 84 which places it i n a queue 188. The ACD 
could thereafter place the telephone call for the sales representative 190. 

However, the ACD is also connected via 1 86 to the network I 6. This 
enables either the queue information 1 88 of the ACD to be communicated to 
the server 20 and displayed by the server 20 to the more than one second 
computers in the sales center 1 88 and the user computer 12. 166. 
Alternatively, the call requests from the network 16 that go to the server 20 be 
communicated to the ACD. Having the ACD connected to the Internet in this 
manner, allows the system to use the sophisticated aspects of an automated 
ACDs. such as queue service, to supplement or substitute for the computer 
queue functions or other similar functions in the server 20. 

In this embodiment, a nrewall 1 92 is also disposed between the user 
computer 12 and the server 20. The firewall 192 can be configured so as to stop 
all incoming traffic from the network 16. stop all outgoing traffic from the user 
computer 12 or any variation there between. Typically, the firewall 192 will 
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allow certain information to pass, but restrict the use of that information. For 
example, the firewall will allow applications in from the network 16. but once 
disposed the user computer 1 2 the application dowqiloaded from the network 
1 6 will not be allow^ed to function upon documents disposed on the user 
5 computer 12 that are not generated from the same server as was the application. 
Such a restriction disables the user applet 22 from modifying interactive forms 
within user computer 12. 

The user computer 12 can expressly permit the function of the user applet 
22 and the server 20 in each individual instance. In ihe preferred embodimeni. 

)() however, the user applet 22 functions are achieved by talkm^ throuiih iIk" 
fn ewall at port 80 using HTTP protocol. As it should be obvious to those 
skilled in the art, this port is usually left open by a firewall for communicaiion 
over the worldwide web. It should be also obvious to those skilled in the an 
that other pons can be used with other modifications or any intermediar\ 

15 solution can be found wherein the user computer 12 grants permissions to the 
user applet 22 to perform functions. 

In the preferred embodiment, however, the u,ser computer 12 as is typical 
is allowed to connect lo an external port of a random server as long as it is 
using HTTP protocol and as long as they arc connecting to port 80 of another 

20 server. Using these specifications, the system performs HTTP tunneling. 

Accordingly, in the preferred embodiment the user applet 22 is sent from port 
80 communicating in MTTP protocol to look very similar to a Common 
Gatewav Interlace ("CGr") request, and is allowed in bv the firewall. 
Thereafter, communication from the user applet 22 still has to be in a protocol 

25 capable of going through the firewall. As such, any request from the user 

applet 22 in this embodiment looks like a standard HTTP request from a web 
page and as such looks like a CGI request. The firewall will let the request 
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through from the user applet 22 onto the network 16. The request is in the 
form of a function caller request, which is transmitted as a message to a CGI 
handler. The CGI handler translates it, passes it to the CGI programmer w-hich 
translates the CGI request to the server 20, wliich in the preferred embodiment 
is a Java server. Any transmission back from the Java server 20 to the user 
applet 22 is done through the same mechanism. Hssentially, the 
communication between the server 20 and the user applet 22 is cloaked to look 
as is it is a request from a web site on the Internet and is such is allowed to go 
through the firewall 192. 

l(J It should be obvious to those skilled in the an that using cither a dilTerent 

server or different handlers or other known methods to go through the llrewall. 
including any pon or other configuration, is possible without departing from 
the scope of this invention. 

Turning now to Figure 7A. which shows one method by which the user 
1.^ computer 12 can enter a call session with the second computer 24 and thereby 
providing audio communication across the network 10 or the telephone 
network in conjunction with visual communication across the network 16. As 
is previously described, the user computer 12 can enter into a call session with 
the second computer 24 by numerous methods. 

20 In Figure 7 A the request to enter a call session starts 200 when a user of 

the user computer 12 is instructed by the sales representative of the second 
computer 24 via telephone to go to a specific web page 202. Displayed on that 
web page will be a sign-on page wherein the uset is prompted to enter 
attributes, such as the users name or any other attributes previously described. 

2> In this embodiment, as the user and the sales party are already in audio 

communication, the user will already have been provided by the sales 
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representative a password that uniquely identirics to the server 20 the session 
the user eompuier 12 is supposed to enter. As such, the user computer 12 is 
also prompted to insert a password 206. After the user enters the password, the 
server 20 places the call request in computer queue 208 as was previouslv 
5 described. 

At this point the server 20 also transmits to the user computer 12 a user 
applel 22. The user applet 22 generates a child browser in the user view of the 
user computer 12 and then begins lo poll the server 20 for call request status 
212. If the call request status is not simultaneously activated by the service 
applet 30. then the user computer 12 is displayed the on-hoid information 2 1 (r 
The on-hold information will include, for example, the user compuler\s status 
m the computer queue, any adveniscmcnts as previously described, and other 
additional inlormation as required. 



tu 



ler 



The control view can also be selectively displayed by the user compult 
1--^ 21 S. However, in the preferred embodiment as the control view is not framed 
wiihm the user interface. As such, the conirol view will be sitting under the 
child browser, and the child browser will display the on-hold information. 

The user applet 22 will continue to poll the server 20 lor the call request 
status 220. If the caller request status is that the user computer is still in the 

20 computer queue, the user applet 22 will update the computer queue information 
still in the computer queue 222. then the user applet 22 will collect the updated 
computer queue information 224 and redisplay the on-hold information in the 
child browser 216. If. however, the second computer 24 has selected the call 
request of the user computer 12, the service applet will notify the server 20 that 

25 the user computer having that session identifier can now be entered into a call 
session 224. After the parties are entered into the call session the 
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commiinicatit)n between ihem and the manipulation of the shared eonstani on 
the user interface of the user computer 12 and the user interface of ilie second 
computer 24 will continue as previously described. 

furning now to Figure 7B, which shows a second method by which the 
5 user computer 12 can enter a call session and start 200 the visual interaction 

between the user computer 12 and the second computer 24. In this method it is 
assumed that the user and the sales representative arc not initially in audio 
communication. Instead, it is assumed that the user is on the worldwide web 
and is viewinu the web page 228 of the sales organization. 

If the user of the user computer 12 wishes to enter the session the user can 
click the call request button 230 on the web page of the sales organization. 
Upon clickmg the call request button 230. the user computer 12 displays on its 
browser a form prompting the user to insert attributes 232 such as name, 
interest, and so on. After the user computer enters the attributes 232 the 
15 attributes are communicated through the network 16 to the server 20. 

whereupon the server 20 passes the attributes 232 via metiiods commonly 
known in the art. The server 20 then goes through skills based routing 234 as is 
previously described and determines which computer queue is appropriate for 
the call request. 

20 The server 20 then places the call request in the computer queue 236. The 

call request in the computer queue can be communicated to the ACf) or 
displayed on the computer queue of the second computer and the user 
computer. The ACD can also determine which qifeue is appropriate lor the call 
request in place of the server and transmit the corresponding selection of the 

2^ computer queue to the server. 
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Regardless, the server then trajismits the user npplct 238 lo the user 
computer. The user applet spawns the child hrowscr and displays Ihe on-hold 
information 240 to the user. Again, the browser of the user computer 12 can 
selectively display the control view 242 ,n a framed method or behind the child 
5 browser. Similarly, the on-hold information can include no. only information 
regarding the call request status in the queue but also the advertisements 
determined either based on the attributes or just the advertisements for the sales 
representatives- company in general, as is previously described. In one 
implementation if the ACD will place the call, the queue of the ACD would be 
10 displayed as communicated lo the server, while m another implemeniat.on. the 
computer queue of the scrA'er would be displayed. It should be obviou.s that the 
queue of the ACD and the computer queue of the server can supplant each 
other or work in conjunction with each other as is necessar> . 

While the browser of the user computer 12 is displaving the on-hold 
5 information, the user applet 22 continues to poll the server 20 for the call 

request .status 244. If the user applet 22 ,s informed by the server 20 that the 
call request is still on the queue 246 then the user applet 22 will collect the 
updated on-hold information 248 and redisplay the on-hold information in the 
child brow.ser 240. This process will continue until the user applet 22 polls the 
0 server 20 for the call request status 244 and is told that the service applet has 
enabled the call request of the customer and informed the server 20 of the 
same. Alternatively, the ACD can place the call and inform the .server that the 
call request has been enabled. At this time, the user applet receives notification 
that the user can enter the call session 250. 

The invention may be embodied in other specific forms without departiui. 
from the spirit or essential characteristics thereof. The present embodiments 
are. therefore, to be considered m all respects as illustrative and not l-estric^i^ e. 
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the scope of the invention being indicated by the appended claims rather than 
by the foregoing description, and all changes which come within the meaning 
and range of equivalency of the claims arc therefore intended to be cmbraeed 
therein. 
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Claims 

The einbodiments of the invention in which an exclusive properly or 
privilege is claimed are deilned as follows; 
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1. A method of providing visual interaction across a network between at least one 
user computer having a browser for reading files and at least one second 
computer in communication with a server, the server interposed between the at 
least one user computer and the at least one second computer, the method 
comprising the steps of: 

establishing a link between the at least one user computer and the ser\'er; 

selectively transmitting a user applet to the at least one user computer, the user 
applet enabling the at least one user computer to communicate with the 
server; 

generating a user view on the at least one user computer; 

communicating that the link is established to a service applet disposed in the at 
least one second computer, the service applet adapted to enable the at 
least one second computer to communicate with the server; 

establishing a session between the at least one user computer and the at least 
one second computer; 

enabling the at least one second computer to display a service view containing 
a representation of the user view; 

selectively manipulating the representation of the user view by the at least one 
second computer; 

communicating to the service applet changes in'the representation of the user 
view; 
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transmitting the changes in the representation of the user view from the service 
applet to the user applet; and 

implementing any changes communicated to the user applet in the user viev^^ 
thereby altering the user view to confomi to the representation of the 
user view of the at least one second computer, thus enabling the at least 
one second computer to communicate to the at least one user computer. 
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A method according to claim 1 further comprising the steps of: 

selectively manipulating the user view by the at least one user computer: 

communicating to the user applet change in the user view; and 

transmitting the change from the user applet to the service applet, thereby 

allovvnng the at least one user computer to control communication with 
the at least one second computer. 
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3. A method according ,o claim 2 wherein ,hc at least one second computer can 
disable an ability of the at least one itser coniputer to selcct.vely manipulate 

the user view. 

4. A method according to clami 1 wherem the user applet and the service applet 
bi-directionaliy communicate. 



5. 



6. 



7. 



A method according to clatm 1 wherein each of the at least one user computer 
is adapted to make call requests to the server, thus requesting vo.ce 



communication. 



A method according to claim 5 funher comprising an automated call 
distributor in communication w,th the server havmg a. least one queue adapted 
lo receive the call requests from the server. 



A method accordmg to cla.m 6 funher comprising at least one computer queue 
disposed on the server adapted to receive notification from the automated call 
distributor of call requests and update the at least one computer queue. 

A method accordmg to claim 5 wherein the server further comprises at least 
one computer queue disposed on the server adapted to selectively receive the 
call requests routed by server, 

A method according to claim 5 further comprising: 

at least one computer queue disposed on the server adapted to receive the call 
requests selectively routed to the at least one computer queue by the 
server; and 

an audio module disposed on the at least one second c'6mputer adapted to 

enable each of the at least one second computers to access the at least 
one user computer having the call requests in the at least one queue. 

A method according to claim 1 funher comprising; 
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al least one computer queue disposed on the server having the call requests: 
and 



queue information representative of the call requests disposed in the at least 
one computer queue, the queue information displayed on the at least 
one second computer. 
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II. A method according to claim I 0 wherein the server updates the queue 

information that is displayed on the at least second computer in real time. 

1 2. A method according to claim 1 0 wherein the server transmits to the at least 
one user computer supplemental information while the call requests of the at 
least one user computer is in the at least one computer queue. 

1 3. A method according to claim 12 wherein the supplemental information is 
selected according to attributes dictated by the at least one user computer. 

14. A method according to claim 1 further comprising the steps of 

extracting from the at least one user computer user attributes representative of 
the at least one user computer; 

ascertaining service attributes of the at least one secured computers; and 

selectively communicating that the link is established to selected ones of the at 
least one second computers by comparing the user attributes and the 
service attributes. 
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15. A method according 10 claim i funher comprising the step of selectively 
adding an additional computer to the session by the server. 

16. A method according to claim 15 wherein additional computer is disposed 
remotely from the at least one user computer and the at least one second 
computer. 

17. A method according to claim 1 wherein the session is password protected. 

18. A method according to claim 1 wherein the service view funher comprises a 
script frame adjacent to the representation of ihc user view and having a 
representation of one or more scripts disposed therein, each of the one or more 
scripts being adapted to communicate with the service applet and being 
adapted to control the representation of the user view. 



10 
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1 ^. A method according to claim I 8 wherein the script initiates a file transfer to 
download information to the user view from a remote location. 

20. A method according to claim 1 8 wherein the script is dynamically defined 
according to attributes of the at least one user computer, 

21. A method according to claim 20 wherein the script is dynamically defined 
according to attributes of the at least one second computer. 



11 



24. 



15 25, 



A method according to claim 20 wherein the script is dynamically defined by 
the second computer to generate a comparison of information within a display. 

A method according lo claim 22 wherein the script is representative of a short- 
cut to enable the at least one second computer to push predetermined 
information to the at least one user computer. 

A method according to claim 1 wherein the any changes to the representation 
of the user view is dictated by a change of a address of the user view, the 
service applet communicating the change of address to the user applet. 

A method according to claim 1 wherein the changes to the representation of 
the user view is forced by the service applet which communicates an image to 
the user applet. 

26. A method according to claim 25 wherein the image is transmitted to the at 
least one user computer at predetermined time intervals. 

20 27. A method according to claim 25 wherein the captured image is transmitted to 
the at least one user computer in real time. 

28. A meihf'fH ac^^oT•diT^o tr^ r^l^irp 1^ riiT-th^r /-/^r-t-.T-^T-;^;,-.*^ tu-^ ^r. 

assigning an image address to the image, the image and the image address 
disposed within the server; 



4SDOCID: <WO 9742728A2_(_> 



wo 97/42728 PCTAJS97/07740 

- 53 - 

communicating lo the user applet the image address; and 

displayiHR the image in the user view of the at least one user computer and the 
representation of the user view in the at least one second computer. 
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29. 



A method according to claim 1 whercn the changes to the representation of 
the user view is captured as image changes, where the image changes are 
representative of a portion of the image that has been seJectiveJy manipulated, 
the service applet communjcating the image changes to the user applet. 
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30. A meihod according to claim 1 wherein the user view and the representation of 
the user view is a form having objects, the method further comprising the steps 

of: 

selectively manipulating the objects by the at least one user computer; 

selectively manipulating the objects by the at least one second computer; 

transmitting the objects to the service applet and the user applet: and 

displaying the objects in the representation of the user view and the user view 
thus allowing the user view and the service view to display the form as 
the form is completed. 

31. A method according to claim 30 wherein the step of transmitting the objects 
further comprises the steps of: 

querying the form to ascertain the objects disposed on the form; 

representing the form as an array having arguments representative of the 
objects; 

selectively updating the objects such that the arguments of the array represent 
any changes to the objects; and 

transmitting at least portions of the array to the service applet and the user 
applet. 
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33. 



34. 



A method according to claim 31 wherein the at least portions of the array are 
representative of only the any changes to the objects. 

A method according to claim 30 wherein the form is enhanced by making the 
objects of the form event driven, wherein the step of transmitting the objects 
further compri.ses the objects communicating to (he applet when the objects 
arc selectively manipulated on the form. 

A method according to claim 30 wherein form is enhanced by generating Java 
applets representative of the objects wherein the step of transmitting the 
objects further comprises transmitting from the .lava applets any changes to the 
objects to the user applet and the service applet. 

35. A method according to claim 1 wherein the user view is a software application 
disposed on the server. 

36. A method according to claim 1 wherein the user view is a software application 
disposed on the at least one second computer. 

37. A method according to claim 1 wherein the user view is a software application 
disposed on the at least one user computer. 

38. A method according to claim 1 wherein the user applet generates the u.ser 
view, the user view having a child browser. 



39. 



40. 



A method according to claim 1 further comprising the step of validating the at 
least one user computer by the server prior to establishing a session. 

A method according to claim 1 further comprising a databa.sc adapted to store 
queue and session information. 



41. A method according to claim 1 wherein the server and user applet selectively 
communicate through a predefined port thus enabling communication through 



a firewall. 
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A method according to claim 1 wherein the at least one user compuier requires 
no additional pre-installed executablcs to communicate with the at least one 
second computer. 

A method according to claim 1 wherein the at least one user computer and the 
at least one second computer can disconnect themselves from the session. 

A method according to claim 1 wherein the at least one second computer can 
selectively disconnect another computer from the session. 

A system for enabling communication between a service parly having at least 
one second computer and a user of a first computer, the system comprising: 

a server; 

a user view displayed on the first computer; 

a user applet downloaded to the first computer by the server and in 

communication with the server, the user applet in communication with 
the server and adapted to generate the user view; 

a service view disposed on the at least one second compuier, the service view 
having the user view displayed therein; and 

at least one service applet disposed within the at least one second computer in 
communication with the server, the service applet having of 
functionality required by the at least one second computer to 
manipulate the service view, whereby the service party is adapted to 
communicate to the user by manipulating the user view through the 
service view. ' 



A method for enhancing a resource collection function of a computer in 
communication with a network while enabling the computer to use 
functionality of a resource, the method comprising the steps of: 
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defining functions of the resource thai the user wishes to enable, the funciions 
of the resource being a sub-sei of the functionality of the resource; 

generating an applet representative the functions of the resource: and 

communicating the applet to the user^s computer as the resource collection 
function, thereby decreasing the resource collection function of the 
user^s computer while enabling the functions of the resource the user 
wishes to enable. 

A method of controlling a browser view on a first computer from a script 
displayed separately from the browser view, the script communicating with an 
applet, the applet adapted to generate the browser view, the method 
comprising the steps of 

providing addresses as simplified commands in the script; 

defining the applet to receive the simplified commands as arguments; 

selectively activating portions of the script; 

associating the simplified commands of the portions of the script with the 

applet such that the applet will generate the browser view dictated by 
the portions of the script that are activated; and 

modifying the browser view in accordance with the simplified commands. 
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48. A method according to claim 47 further comprising a remote computer having 
a user view and a user applet in communication with the applet, the user applet 
adapted to receive the simpHfied commands from the applet and modify the 
user view of the remote computer. 

49. A method according to clami 48 wherein the applet has session intormaiion 
representative of the first computer and the remote computer. 
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A method according to claim 48 wherein the step of providing addresses as 
simplified commands in ihc script further comprises the steps of: 

ascertaining user attributes of the remote computer; and 

selectively generating addresses such that the addresses relate to the user 

attributes thus allowing the browser view to contain a display related lo 
the user attributes. 
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51 . A method according lo claim 50 wherein the server selectively generates the 
addresses. 
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^2. A method according to claim 50 wherein the first computer seleciiveiy 
generates the addresses. 
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53. A i-nethod according lo claim 48 wherein the. step of providing addresses as 
simplified commands in the script further comprises the steps of : 

ascertaining session attributes representative of service attributes of the second 
computer and user attributes of the remote computer; and 

5 sclectivelv generating addresses such that the addresses relate to the sessi(Mn 

attributes. 
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